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The  issue  of  early,  rigorous  evaluation  of  program  costs  is  becoming  more  important  as  defense 
funding  comes  under  greater  scrutiny.  Often  at  this  point  in  the  life  cycle,  a  requirement  or 
desired  capability  is  known,  but  the  manifestation  of  the  solution  is  unknown  or  described  only  at 
a  high  level.  Can  capabilities  alone  be  used  to  produce  a  cost  estimate?  If  so,  how  can  we  link 
the  proposed  solution  to  existing  systems  if  only  a  particular  solution’s  general  capability  set  is 
known? 

This  work  submits  that  better  strategic  decisions  within  fiscal  constraints  could  be  made  if  rough 
order  of  magnitude  (ROM)  estimates  were  available  for  proposed  materiel  or  non-materiel 
solutions,  based  on  that  solution’s  capability  set.  This  project  further  proposes  the  use  of  a 
knowledge  base  to  provide  support  for  these  estimates;  it  is  known  as  the  joint  Capabilities 
Knowledge  Base  (CKB).  By  using  the  relevant  entities  extracted  from  CKB,  a  ROM  cost  estimate 
may  be  developed  using  a  wide  spectrum  of  techniques. 

According  to  Department  of  Defense  (DoD)  guidance  dated  June  19,  2006,  the  [2006] 
Quadrenniel  Defense  Review  (QDR)  report  called  upon  senior  departmental  leaders  to  “better 
integrate  the  processes  that  define  needed  capabilities,  identify  solutions  and  allocate  resources 
to  acquire  them  in  order  to  enable  corporate  decision-making  that  cuts  across  traditional 
stovepipes”.  In  response  to  this  directive,  DoD  leaders  are  evaluating  a  new  early  lifecycle 
decision-making  framework  that  includes  a  Concept  Decision  (CD)  Review  (supported  by  an 
Evaluation  of  Alternatives  or  EoA).  The  CD  Process  has  been  set  forth  as  a  way  to  combine 
requirements,  capabilities  portfolio  evaluation,  and  resource  allocation  considerations  in  the 
pursuit  of  joint,  efficient,  and  well-informed  decision-making  early  in  the  acquisition  life  cycle.  The 
Concept  Decision  will  either  replace  or  occur  in  conjunction  with  Milestone  A  to  decide  which  of 
the  prospective  solutions  provided  by  the  EoA  will  best  enhance  overall  US  defense  capability 
while  balancing  priorities  of  cost,  schedule,  and  risk  management. 
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The  issue  of  early  and  rigorous  evaluation  of  program  costs  becomes  more  and  more  important 
as  defense  funding  becomes  more  scrutinized.  Clearly,  decision-makers  need  high-fidelity  cost 
information  at  this  key  decision  point,  but  more  often  than  not,  it  is  scant.  Providing  reliable, 
useful  cost  estimates  very  early  in  the  acquisition  life  cycle  is  challenging  for  several  reasons. 
Often  at  this  point  in  the  life  cycle,  a  requirement  or  desired  capability  is  known,  but  the 
manifestation  of  the  solution  is  unknown  or  described  only  at  a  high  level.  This  is  certainly  a 
challenge,  given  that  defense  cost  estimating  is  usually  performed  given  a  detailed  system 
description.  Given  the  changing  face  of  the  battlefield  and  warfare,  proposed  solutions  are  often 
unlike  anything  presently  in  existence. 

As  any  cost  estimator  can  confirm,  there  exists  a  spectrum  of  situations  in  which  a  cost  estimate 
may  be  prepared.  One  theoretic  extreme  is  creating  a  cost  estimate  in  a  situation  where  there  is 
very  little  information  about  the  item  being  estimated  and  no  supporting  data.  The  other  extreme 
is  when  the  entity  being  estimated  is  fully  understood,  and  all  data  exists  to  estimate  the  cost 
exactly.  In  this  case,  the  data  are  actual  costs  after  the  item  has  been  developed,  constructed,  or 
bought.  Figure  1  shows  these  extrema  along  with  all  points  in  between. 
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Figure  1:  The  Cost  Estimating  Spectrum 


As  we  progress  from  the  point  of  no  information  to  the  point  of  perfect  information,  our  cost 
estimating  methodology  changes  to  suit  the  information  climate.  For  instance,  when  information 
about  the  item  or  service  is  higher-level  and/or  data  is  not  readily  available  (Figure  1,  left),  cost 
estimators  tend  to  rely  upon  analogies  and  parametric  methods  to  produce  their  estimate. 
However,  as  we  move  toward  the  right,  estimates  tend  to  utilize  more  “data-hungry” 
methodologies  such  as  engineering  builds  and  projections  using  actual  costs  to  date.  It  is  also 
clear  to  the  casual  observer  that  as  we  move  along  the  spectrum  from  left  to  right,  we  may  expect 
our  estimate  to  be  more  reliable  and  closer  to  the  actual  cost  at  project  or  acquisition  completion. 

The  pre-Milestone  A  costing  environment  is  particularly  challenging.  This  is  the  stage  in  which 
information  is  often  extremely  scarce.  Figure  2  illustrates  the  “sub-spectrum”  of  pre-Milestone  A 
data  availability. 
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Figure  2:  Spectrum  of  Information  Availability  at  Milestone  A 


Depending  upon  the  situation,  there  may  be  one  or  many  proposed  solutions  to  a  set  of  capability 
gaps  before  Milestone  A.  These  solutions  could  be  a  materiel  system  such  as  a  vehicle  or 
software  package,  or  it  could  be  a  non-materiel  solution,  such  as  a  policy  change  or  a  training 
curriculum  change.  As  one  can  see  in  Figure  2,  the  information  regarding  the  proposed 
solution(s)  could  range  from  simply  the  desired  capability  expressed  in  very  qualitative  terms  to  a 
relatively  detailed,  well-developed  concept  with  some  technical  platform  specifications.  The  most 
commonly-occurring  scenario,  however,  is  nearer  to  the  middle  where  there  exists  high-level 
capabilities  information  along  with  some  very  general  solution  information. 

Since  every  cost  estimate  of  an  item  or  project  must  be  based  on  some  type  of  past  experience, 
pre-Milestone-A  cost  estimating  is  no  exception.  How  can  we  link  the  proposed  solution  to 
existing  systems  (our  past  experience)  if  we  know  only  a  particular  solution’s  general  capability 
set?  Can  capabilities  alone  be  used  to  produce  a  cost  estimate?  If  so,  could  that  cost  estimate 
be  used  in  decision-making  with  any  degree  of  confidence? 

Suppose  we  made  the  assumption  that  a  system’s  capabilities  have  a  relationship  to  its  cost. 
To  the  casual  observer,  this  assumption  seems  rather  logical.  If  we  buy  something  that  can  do 
more,  do  it  quicker,  or  do  it  better,  then  it  should  cost  more.  However,  one  can  identify  situations 
in  which  this  assumption  might  not  hold;  if  a  particular  computer  technology  is  maturing  at  an 
accelerated  rate,  the  cost  to  acquire  that  capability  might  not  be  correlated  to  the  cost  of  acquiring 
a  similar  capability  five  years  ago.  Yet,  even  this  example  has  a  relationship  between  capability 
and  cost  upon  closer  inspection;  to  arrive  at  an  acceptable  cost  estimate  one  must  understand 
the  rate  of  technology  maturation  (and  this  maturity  information  may  or  may  not  be  available  to 
the  analyst).  The  question  at  hand,  however,  is  whether  or  not  capabilities  can  predict  cost  within 
some  acceptable  level  of  percentage  error  to  provide  decision  makers  with  data  that  helps  avoid 
decisions  that  would  yield  negative  future  cost  effects.  In  theory,  these  decisions  could  be 
avoided  if  a  rough  order  of  magnitude  (ROM)  estimate  is  available  that  is  based  on  the  proposed 
materiel  solution’s  set  of  capabilities. 

The  capabilities  costing  team  at  the  Office  of  the  Deputy  Assistant  Secretary  of  the  Army  for  Cost 
and  Economics  (ODASA-CE)  is  currently  tackling  the  challenging  pre-Milestone-A  costing 
environment.  Our  approach  includes  the  use  of  a  knowledge  base  that  records  current  system 
cost  information  and  capabilities.  In  fact,  the  Joint  Capabilities  Knowledge  Base  (CKB)  is 
presently  under  construction.  By  using  the  relevant  entities  extracted  from  the  CKB,  a  ROM  cost 
estimate  may  be  developed  using  a  wide  spectrum  of  techniques. 

Numerous  costing  approaches  are  being  examined  and  developed  as  this  project  evolves,  one  of 
which  follows:  Let  us  assume  that  the  set  of  capabilities  requiring  a  cost  estimate  is  rank- 
ordered;  in  other  words,  we  know  which  capabilities  among  the  group  are  most  critical,  somewhat 
necessary,  or  only  slightly  needed.  Depending  on  whether  the  entities  (that  will  be  extracted  from 
the  CKB)  have  exact  or  partial  matches  in  capabilities,  we  can  then  apply  an  appropriate 
weighting  factor  for  certain  combinations  of  capability  matches;  exact  matches  would  receive  a 
higher  weighting  than  partial  matches,  for  example.  Next,  relevant  entities  are  extracted  from  the 
knowledge  base  that  can  be  used  in  our  cost  estimate.  The  assigned  weightings  are  applied.  If  a 
particular  system  entity  is  deemed  to  be  even  more  relevant  to  the  solution  being  estimated,  it 
may  be  further  emphasized  in  a  variety  of  ways. 


Larger-scale  case  studies  using  realistic  scenarios  are  under  development  to  test  the  usefulness 
and  strength  of  the  methodology  frameworks  being  considered,  which  include  simplistic 
techniques  like  that  described  above  to  more  intricate  parametric  and  data  mining  approaches.  It 
is  important,  however,  to  emphasize  that  cost  estimates  at  this  point  in  the  life  cycle  are  highly 
situation-specific,  and  thus  methodologies  under  development  are  only  recommended  strategies. 
The  analyst’s  judgment  is  a  key  component. 


Martha  Roper  is  an  Operations  Research  Analyst  (ORA)  with  the  Office  of  the  Deputy  Assistant 
Secretary  of  the  Army  for  Cost  and  Economics  (ODASA-CE).  Ms.  Roper  is  leading  early-life 
cycle  cost  analysis  initiatives  in  conjunction  with  analysts  Nancy  Houdek  and  Katherine 
McCormack. 


